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NATIONAL INSTITUTE FOR EXCELLENCE IN TEACHING 


Researchers at NIET and elsewhere have studied the effectiveness of TAP: The System for Teacher and 
Student Advancement (TAP) in raising student achievement, improving the quality of instruction, and 
increasing the ability of high-need schools to recruit, retain and support effective teachers. This document 
provides summaries of some of the key studies on the TAP System to date. 


NIET Mission 


Recognizing that an effective teacher is the most important school-based factor impacting 
student achievement, NIET is committed to ensuring a highly skilled, strongly motivated, and 
competitively compensated teacher for every classroom in America. NIET supports states, districts, 
schools and institutions of higher education in recruiting, developing, supporting, and retaining high- 
quality human capital in order to raise achievement levels for all students. NIET provides both on-site 
and online support across multiple aspects of educator effectiveness through educator evaluation, 
professional development and teacher leadership. This support is centered on NIET’s comprehensive 
educator effectiveness model, TAP™: The System for Teacher and Student Advancement (TAP System) 
and the customizable Educator Effectiveness Best Practices. NIET is using these initiatives to impact 
over 250,000 educators and more than 2.5 million students each school year, and these initiatives have 
yielded consistently positive growth in teacher performance and student achievement. 


TAP: The System for Teacher and Student Advancement Description 


With two decades of implementation in urban, suburban, rural, remote, tribal, public, private, and 

charter schools, the TAP System is impacting educators and students across the country. The TAP 

System creates multiple career paths for teachers, provides ongoing applied professional development 

using a rigorous rubric of evaluation, and provides performance-based compensation to teachers and 

administrators. Each of these core elements is briefly discussed below. For more information, visit 

www.niet.org. 

= Multiple career paths. In TAP schools, skilled teachers have the opportunity to serve as master and 
mentor teachers, receiving additional compensation for providing high levels of support to career 
teachers and increasing instructional effectiveness across the faculty. Master and mentor teachers 
form a leadership team, along with administrators, to deliver school-based professional support and 
conduct evaluations with a high level of expertise. 

= Ongoing applied professional growth. Led by master and mentor teachers, TAP teachers participate 
in weekly meetings where they examine student data, engage in collaborative planning, and learn 
instructional strategies that have been field-tested in their own schools. Teachers benefit from a 
national TAP database of instructional strategies and their colleagues’ experiences. Professional 
development continues in the classroom as master teachers model lessons, observe classroom 
instruction, and support teachers’ pedagogical improvement. 

=  Instructionally focused accountability. TAP teachers are observed in classroom instruction several 
times a year by multiple trained observers, including principals and master and mentor teachers, using 
rubrics for several dimensions of instructional effectiveness. Evaluators are trained and certified, and 
leadership teams monitor the reliability and consistency of evaluations in their schools. These 
classroom evaluations are complemented by analyzing student achievement growth, rounding out a 
multi-measure system of teacher evaluation. Evaluation results are used as formative feedback in 
one-on-one mentoring sessions, and guide planning for individualized professional development. 


=  Performance-based compensation. TAP teachers have the opportunity to earn annual bonuses based 
on their observed skills, knowledge, and responsibilities, their students’ average achievement growth, 
and school-wide achievement growth. Master and mentor teachers receive additional compensation 
based on their added roles and responsibilities, and principals can earn additional compensation 
based on school-wide achievement growth and other measures of effectiveness. 
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TAP Impact in Schools, Districts and States 


The TAP System stands out because of its more than two decade-long track record of growth and 
success in raising student achievement in high-need schools. The research evidence reveals several 
key reasons for TAP’s positive impact: an evaluation system capable of differentiating teacher 
performance levels and providing detailed feedback for improvement, ongoing professional growth in 
classroom practice using student and teacher data to guide improvement, recruitment and retention 
of effective teachers, and the creation of a challenging, rewarding, and collegial environment focused 
on high-quality instruction and student learning. 


With over 50 studies, the TAP System is strongly supported by an expanding preponderance of 
performance data that comes from examining the impact of the system across multiple locations and 
conducted by different researchers using varied methodological frameworks, including propensity score 
matching and well-designed baseline equivalency studies. The next section discusses several of these 
studies demonstrating the impact of the TAP System. In what follows, we provide a one-page summary 
of 10 key TAP System studies and their findings. Selected studies demonstrate evidence of the TAP 
System’s impact on student achievement and teacher retention. Additionally, studies provide evidence 
of strongly positive attitudinal data from teachers and principals implementing the TAP System. 
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Student Achievement 
Nationwide — Multi-state 


Barnett, J. H., Hudgens, T. M., & Logis, H. A. (2017a). TAP research summary spring 2017: Examining 
the evidence and impact of TAP: The System for Teacher and Student Advancement. Santa 
Monica, CA: National Institute for Excellence in Teaching. 


The authors examined student achievement growth in 353 TAP System schools and 3,870 control schools. 
The TAP System, a growing number of state education agencies, and many researchers use a statistical 
method called “value added” or "growth modeling" to measure the influence of teachers and schools on 
student achievement during a school year. Value added tells how much the school and teacher have 
contributed to student learning compared to other schools and teachers in the same state with similar 
students. In TAP, student learning growth is measured on a 1-5 scale: 


5: Much more than a year’s growth (significantly higher than average at about the 95% confidence level) 
4: More than a year’s growth (significantly higher than average at about the 70% confidence level) 

3: One year’s growth (indistinguishable from the average) 

2: Less than a year’s growth (significantly lower than average at about the 70% confidence level) 

1: Much less than a year’s growth (significantly lower than average at about the 95% confidence level) 


Drawing from raw school-level, composite math and reading score data provided by SAS® the authors 
examined the percentage of 1) control schools, 2) new TAP schools, and 3) continuing TAP schools earning 
one year or more of academic growth (scoring at least a school-wide value-added score of “3”). 


The authors found gains in student achievement in TAP System schools. More importantly, these 
improvements increase over time (Figure 1). 
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Figure 1. Student achievement growth by TAP System status. 
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Nationwide — Multi-state 


Hudson, S. (2010). The effects of performance-based teacher pay on student achievement. Discussion 
Paper for the Stanford Institute for Economic Policy Research, Stanford University. 
Retrieved from: http://www-siepr.stanford.edu/repec/sip/09-023.pdf 


In 2010, Hudson examined the effect of the TAP System on student achievement across 151 schools in 
10 states (Arkansas, Colorado, Indiana, Minnesota, North Carolina, Ohio, Pennsylvania, South Carolina, 
Tennessee, and Texas). The author used a statistical control matching method to ensure that the TAP 
schools and the comparison schools were equivalent prior to the intervention being implemented. 
Hudson also used a differences-in-differences approach to further account for any differences between 
the groups and to ensure that the evaluation was able to isolate the impact of the program. 


Results of the study indicate that students in TAP schools outperformed students in comparison schools 
by approximately 0.15 standard deviations in mathematics, and smaller effects but in favor of the TAP 
schools in reading (Table 1). Hudson explains these findings in context to other education interventions 
by noting that “the estimated effect of TAP on mathematics achievement is more than twice as large [as 
class size reduction effects]” (p. 28). Overall, this study provides an evidence that performance-based 
pay for teachers can affect student achievement. 


Table 1. Results of difference-in-differences estimation 


Reading Mathematics 
Diff-in-Diff 0.109* 0.127* 
(0.065) (0.075) 
Detrended Diff-in-Diff 0.134*** 0.152** 
(0.068) (0.083) 
One Year Lag 0.051 0.218** 
(0.074) (0.086) 
Two Year Lag 0.076 0.244** 
(0.098) (0.108) 


* p< 0.1. ** p<.05. *** p< 0.01. 


Notes: Standard errors are clustered at the school-level and presented in parentheses. All specifications 
include the full set of demographic controls along with year, state, and grade fixed effects. 
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Nationwide — Multi-state 


Solmon, L. C., White, J. T., Cohen. D., & Woo, D. (2007). The effectiveness of the teacher advancement 
program. Santa Monica, CA: National Institute for Excellence in Teaching. Retrieved from: 
http://secure.niet.org/pubs/effective_tap07_full.pdf 


In 2007, Solmon et al. analyzed the impacts of the TAP System in terms of value-added gain scores across 
650 classrooms in six states, including 61 TAP schools and 285 control schools. The authors analyzed the 
student achievement gains at two levels of comparison—teacher-to-teacher and school-to-school. To 
evaluate TAP teachers (and similarly in evaluating TAP schools), researchers calculated the effect of each 
teacher on student progress as assessed by the difference between the actual average scores of the 
teacher’s students and the expected average scores of those students (as derived from previous scores). 
Through this process, the authors created a statistical control group for the TAP teachers based on 
performance. 


Results of the study indicate that in every state more TAP teachers demonstrated statistically significant 
at or above average amount of student achievement growth than control group teachers. Specifically, all 
states had a smaller percentage of TAP teachers scoring a “1 or 2” than control teachers, meaning fewer 
TAP teachers had students whose progress was below average. Additionally, all states had a higher 
percentage of teachers who scored a “3 or above” than their controls, meaning more TAP teachers were 
significantly more effective in raising their students’ scores than control teachers. Further, TAP schools 
outperformed their controls in 57% of the categories (1 -5, by state) in math and in 67% of the categories 
in reading (see Figure 2). 
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Figure 2. Percent of comparisons in which TAP schools outperform control schools. 
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Indiana 


Barnett, J. H. & Wills, K. C. (2016). Narrowing the gap: Achievement gaps in Indiana TAP schools. 
Paper presented at the annual meeting of the American Educational Research Association. 
Washington, D.C. 


Barnett and Wills (2016) examined the impact of the TAP System on Black-White and Hispanic-White 
achievement gaps in Indiana schools. Drawing from Indiana Statewide Testing for Educational Progress 
(ISTEP) data for 2009-10 through 2013-14, the authors calculated Black-White achievement gaps as 
differences between Black and White students’ passing rates, and Hispanic-White achievement gaps as 
differences between Hispanic and White students’ passing rates. 


First, the authors compared achievement gaps in 32 TAP schools that began implementing TAP in school 
year 2010-11 with other Indiana schools. Twenty TAP schools had sufficient Black and White enrollment 
to report data for Black and White students for 2009-10 and 2013-14, and 24 TAP schools had sufficient 
Hispanic and White enrollment to report data for Hispanic and White students for these school years. 
Study results indicate TAP schools have narrowed the Black-White gap in ELA and Hispanic-White gap in 
ELA and math more than other Indiana schools have. 


Second, the authors selected propensity score matched control schools separately for 1) the 20 TAP 
schools reporting data for Black and White students in 2009-10 and 2013-14, 2) the 24 TAP schools 
reporting data for Hispanic and White students in 2009-10 and 2013-14. They matched on percentage of 
students (all races/ethnicities) passing both ISTEP ELA and math in 2009-10, grade span of the school, 
percentage of students receiving free or reduced-price lunch, and percentage of Black or Hispanic 
students. Figure 3 shows that TAP schools outperformed their matched schools; while achievement gaps 
at the TAP schools narrowed (positive values), most gaps at the matched schools widened (negative 
values). 
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Figure 3. Changes in achievement gaps, compared with propensity score matched controls. 
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Louisiana 


Barnett, J. H., Wills, K. C., & Kirby, P. C. (2014). Comprehensive educator effectiveness models that 
work: Impact of the TAP System on student achievement in Louisiana. Santa Monica, 
CA: National Institute for Excellence in Teaching. Retrieved from: 
http://www. niet.org/assets/Publications/louisiana-tap-student-achievement.pdf 


In 2014, Barnett, Wills, and Kirby evaluated the impact of the TAP System across 66 schools in Louisiana 
using two rigorous analytic strategies. First, the authors employed a linear regression to compare 2012- 
13 Assessment Index (Al) performance of the 66 TAP schools and non-TAP schools statewide, controlling 
for prior (2010-11) achievement, percentage of students receiving free or reduced-price lunch, school 
configuration, school size (number of students), and percentage of English language learners. Controlling 
for the covariates, implementation of the TAP System showed a significant positive effect on 2012-13 
achievement: the 66 TAP schools scored 3.7 points higher on average than non-TAP schools (p < .01). 


Table 2. Coefficient estimates for regression with 2012-13 Louisiana Assessment Index 


Estimate Standard error t p 
Intercept 19.252 3.558 5.41 .000*** 
2010-11 Al 0.786 0.022 36.16 .000*** 
% FRL -0.183 0.022 -8.27 .000*** 
School configuration (baseline is elementary): 
Middle/junior high school -1.669 0.786 -2.12 .034* 
High school -4.009 0.984 -4.07 .000*** 
School size -0.003 0.001 -2.37 .018* 
% LEP 0.334 0.076 4.41 .000*** 
TAP school 3.699 1.221 3.03 .003** 


* 0 <.05. ** p<.01. *** p< 001. 


Second, the authors compared the 66 TAP schools with a propensity score matched group of non-TAP 
schools. The average 2012-13 Al for TAP schools (64.45) was 5.47 points greater than the average for the 
matched controls (58.98). To illustrate the gain in score for the TAP schools, Figure 4 shows the equivalent 
starting point for both TAP and control schools as a function of the maximum score on the 2010-11 Al 
(maximum of 200) and the change in score relative to the maximum score on the 2012-13 Al (maximum 
of 150). The score comparison indicates the TAP schools made larger gains in student achievement than 
matched comparison schools. 
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Figure 4. 2012-13 K-8 Assessment Index averages. 
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Louisiana 


Mann, D., Leutscher, T., & Reardon, R. M. (2013). The system for teacher and student advancement: 
An evaluation of achievement and engagement in Louisiana. Santa Monica, CA: National 
Institute for Excellence in Teaching. Retrieved from: 
http://www.niet.org/assets/Publications/interactive-louisiana-student-achievement.pdf 


In 2013, Mann, Leutscher, and Reardon conducted a third-party evaluation of the impact of the TAP 
System across 15 schools in Louisiana. In order to determine impact, a one-to-one nearest-neighbor 
matching algorithm with replacements was created to find a comparison school for each TAP 
school. Based on the propensity scores computed using the selection model, the algorithm selected the 
non-TAP school with the propensity score closest to the propensity score of the TAP school. 


There was no significant difference between the TAP schools and their matched comparison schools 
in the pretreatment year, t(26) = 0.080, p > 0.05. However, as shown in Figure 5 below, the authors 
found that the TAP schools’ School Performance Scores (SPS) increased much more than that of their 
matched schools. Specifically, TAP schools significantly outperformed comparison schools after two 
years (F(1, 26) = 6.37, p < 0.05) and three years of implementation (F(1, 24) = 5.30, p < 0.05). 
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Figure 5. School Performance Score trends for TAP schools and matched schools. 
Results of the study also indicate that in the four primary subjects assessed, there was a significant effect 


in favor of the TAP schools for ELA: F(1, 6421) = 6.334, p = 0.012; Mathematics: F(1, 6421) = 86.386, p< 
0.001; Science: F(1, 7084) = 31.792, p < 0.001; and Social Studies: F(1, 7085) = 87.411, p < 0.001. 
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Texas 


Barnett, J. H., Hudgens, T.M., & Logis. H. A. (2017b). Evaluation of the TAP System in Texas. Paper 
presented at the 17th Annual National TAP Conference. New Orleans, LA. 


Barnett, Hudgens, and Logis (2017b) examined the impact of TAP in 46 schools serving K-8 grade 
students in Texas. Specifically, they tested whether TAP schools outperformed similar non-TAP schools in 
math and reading state assessments. In order to determine impact, the authors performed a one-to- 
one propensity score analysis to find a comparison school for each TAP school that had similar school 
performance in the pretreatment year and the same school configuration. Using the sample of 46 
TAP schools and 46 matched schools in Texas, Figure 6 below shows that over the duration of 
TAP implementation, TAP schools outperformed matched schools by an average of 0.12 standard 
deviation in math and reading assessments combined. 
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Figure 6. School performance gain in standard deviation units in 46 TAP schools versus 46 matched schools. 


In a second analysis, the authors refined the sample by selecting a subset of TAP schools that met the 
following criteria: 1) had teachers who reported strong levels of support on key elements of TAP and 2) 
did not experience any dramatic changes in school demographics. These criteria yielded 29 TAP schools. 
Figure 7 below shows that throughout the implementation of the TAP System, these 29 TAP schools 
outperformed matched schools by an average of 0.38 standard deviation (SD = 1.20, t(28) = 1.69, p = 0.10) 
in math and reading assessments combined. 
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Figure 7. School performance gain in standard deviation units in 29 TAP schools versus 29 matched schools. 
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Arizona 


Schacter, J., & Thum, Y. M. (2005). TAPping into high quality teachers: Preliminary results from the 
Teacher Advancement Program comprehensive school reform. School Effectiveness & School 
Improvement, 16(3), 327-353. doi:10.1080/13803610500146202 


Schacter and Thum (2005) examined the impact of TAP on student performance and teachers’ attitudes 
in four schools in Arizona. To examine student achievement, for each of the four TAP schools, one or more 
non-TAP comparison schools were selected based on achievement, school size, percent of students 
eligible for free lunch, school grade level configuration and location through a statewide cluster analysis 
conducted by the Arizona Department of Education. To account for differences, the authors controlled for 
initial achievement. Additionally, the authors employed all available covariates and compared growth 
rates, instead of final attainment, to equate groups as carefully as possible. 


Using a multivariate-multilevel model, the authors found that in the aggregate, TAP schools gained 
significantly (p < .05) more than control schools in reading, mathematics, and language achievement in 
both the 2000-2001 and 2001-2002 school year (Table 3). Specifically, TAP schools significantly out-gained 
comparison schools by approximately 30% over two years (effect sizes of .35 and .41 in 2001 and 2002, 
respectively). Additionally, the authors found that the level of TAP System implementation played a role 
in each school’s success — the school with the highest levels of implementation performed the best, while 
the school with the lowest levels of fidelity performed the worst. 


Table 3. Comparison of TAP school gains to controls 


2000 - 2001 2001 - 2002 
Gains Gains 
Mean _ std Effect Mean std Effect 
All TAP Schools Gains Compared to Controls 2.29 0.56 35 2.68 0.60 41 
TAP 1 -0.47 1.49 -.07 -0.29 1.40 -.04 
TAP 2 3.88 1.09 .60 2.86 1.22 44 
TAP 3 4.12 1.28 .63 13.70 1.47 2.10 
TAP 4 1.47 0.78 .23 -4,10 0.96 -.63 


The authors also examined survey responses from an instrument designed to assess teacher job 
satisfaction with the principals of the TAP System. The authors found that teachers reported support for 
the four core elements of TAP (multiple career paths, ongoing professional development, instructionally 
focused accountability, and performance-based compensation). Additionally, the majority of teachers 
reported feeling a strong sense of collegial support and teamwork within their schools. 
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Teacher Retention 


Barnett, J. H., & Hudgens, T. M. (2014). Staying power: The impact of the TAP System on retaining 
teachers nationwide. Santa Monica, CA: National Institute for Excellence in Teaching. 
Retrieved from: http://www.niet.org/assets/Publications/staying-power-tap-system- 
teacher-retention.pdf 


Barnett and Hudgens (2014) drew upon a sample of 12,095 teacher-level records, representing 413 
schools in 10 states, to examine teacher retention rates in schools that implemented the TAP System 
during the 2010-11, 2011-12, and 2012-13 school years. Specifically, retention rates were examined 
across three types of teacher groups, those who: 1) taught continuously at the same TAP school (“TAP 
school stayers”); 2) transferred from one TAP school to another TAP school (“TAP school movers”); and 3) 
left TAP schools altogether (“TAP school leavers”). 


For 2010-11 to 2011-12, the percentage of stayers by state ranged from a low of 92.7% to a high of 99.2% 
with an average value of 93.9%. From 2011-12 to 2012-13, the percentage of stayers by state ranged from 
a low of 91.2% to a high of 97.3% with an average value of 94.5%. With an overall average retention rate 
of 94%, this finding exceeds the national figure for all schools and for high-need schools (Figure 8). Given 
that 97% of TAP schools are high-need schools, the comparison to the national average and high-need 
school average is appropriate. Additionally, administrators echoed the value of the TAP System’s impact 
on teacher retention — 94% of principals reported that the TAP System positively impacted their ability to 
retain effective teachers. 
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Figure 8. Retention rates in TAP schools relative to national average and high-need schools. 


Beyond retaining more teachers, the teachers in the TAP System schools demonstrate growth in their 
instructional ability as measured by the gain in their SKR scores. More importantly, this growth in 
instructional ability is also occurring while student performance scores are demonstrating growth, as 
measured by the gain in the classroom value-added scores. The gain in scores is found for both TAP 
teachers who stay in their school as well as for teachers who move to another TAP school. This finding of 
the TAP boost for instruction and student achievement for those who transfer to another TAP school 
provides evidence that the TAP System, as a comprehensive educator effectiveness model, works 
effectively across sites and within sites. 
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Educator Attitudes 


Barnett, J. H., Hudgens, T. M., & Alexander, J. L. (2017). 5,000 Strong: Teachers’ Responses to a New 
Educator Effectiveness and Support System. Paper presented at the annual meeting of the 
American Educational Research Association. San Antonio, TX. 


The National Institute for Excellence (NIET), which oversees the implementation of TAP, recognizes the 
importance of educator views and attitudes in creating a successful reform. Each year, NIET surveys 
educators to understand their perceptions of the TAP System and its impact. As a result of this effort, NIET 
has collected results from a survey of over 10,000 educators from across the nation for over a decade. 


Barnett, Hudgens, and Alexander (2017) find that teachers overwhelmingly support the core elements of 
the TAP System and the collegiality that the TAP System creates within their schools. Additionally, support 
for these elements has increased over time. Figure 9 displays teacher reported support, comparing an 
early adoption year (2005) to the current year (2016). 
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Figure 9. Level of reported support for TAP core elements from teachers nationwide, over time. 


The substantially positive results from the TAP teacher survey are echoed by the TAP principal survey. As 
shown in Figure 10, principals have overwhelmingly reported that TAP has a positive effect on 
instructional practices, collegiality, student achievement, and teacher effectiveness. 
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Figure 10. Administrator survey results on TAP outcomes in their schools, 2016. 
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